Summary

• Measurement-first LLM-pipeline review. Live eval:rag (44 cases) priced at current gpt-5.5 rates ($5 / $0.50 cached / $30 per 1M in/cached/out) + rag_retrieval_logs telemetry (306 rows) validate the fast→quality-gate→strong cascade against the current model generation, so no routing/timeout/schema churn is made. Fast and strong are the same model (gpt-5.5) differentiated only by reasoning effort; strong costs 1.8× fast ($0.101 vs $0.056) at +13s median latency while fast grounds 100% of the queries routed to it. Routing everything to a strong model would roughly double cost and latency on the ~40% of traffic fast handles perfectly, for no quality gain.
• Item 2 was already done or not justified: citation→retrieved-chunk-ID enum (answerJsonOutputSchemaForResults) already closes the master plan's "where practical" gap; strongReasoningEffortForQueryClass already caps effort except for safety-critical classes; prompt caching is already configured. Only change here is the answer cache key v17→v18 (the cached instruction prefix changed).
• Item 3 — prompt-injection hardening from docs/rag-injection-threat-model.md (new §6): provenance/untrusted-data boundary in answerInstructions; neutralize the previously-RAW title/file_name/image caption/tableTitle/tableLabel/index-warning fields; case-insensitive fence escaper + neutralize-after-glyph-normalization; escape forged fence sentinels in every derived field in place (closes Vector E without the ~940 tokens/answer a per-field wrapper added). New tests/rag-injection.test.ts lands the deterministic INJ-3/4/6/12 cases + neutralizer/escaper regression guards.
• Incidental cost effect: the longer instruction prefix now crosses OpenAI's prompt-cache minimum, so cached-input share rises from ~4% to ~40–80% across runs and cost/answer is neutral-to-lower ($0.038–0.030 vs $0.040 baseline) despite the added text.

Cost-per-answer by route (BEFORE / main, 44 cases, gpt-5.5 pricing)

Before / after (44-case eval:rag)

The 1-case eval:rag delta is within run-to-run noise: the two residual "regressions" are strong-route answers at 44s and 55s latency (timeout fallbacks on cases already latency-fragile on main), one baseline-failing clozapine case actually recovered to grounded, and eval:quality --rag-only on the same code matched baseline at 28/30. No content-level grounding loss from the hardening.

Verification

• npm run verify:cheap — green (1261 tests, lint 0 errors, typecheck clean)
• npm run verify:ui — n/a (no UI/routing/styling change)
• npm run verify:release — not run (no release/handoff claim)
• npm run format:check — changed files pass Prettier
• npm run eval:retrieval:quality — retrieval is not touched (prompt-assembly + content-sanitizer only); baseline on main recorded 36/36, failed_cases=0 before any change. Not re-run because ranking/selection/chunking/scoring are unchanged.
• npm run eval:rag (full) + npm run eval:quality -- --rag-only — grounded-supported 0.9333 (= baseline), invalid-citation 0. The non-zero citation/numeric/latency threshold flags in the quality report are pre-existing on main (identical failing cases in the pre-edit eval:rag baseline: clozapine-anc-withhold-threshold, community-admission, illegal-substances), not introduced here.
• npm run check:production-readiness — n/a (no clinical-workflow/privacy/env/Supabase/governance/deployment change)
• npm run check:deployment-readiness — n/a

Clinical Governance Preflight

• Source-backed claims still require linked source verification before clinical use — grounding gates, numeric verification, and citation validation are unchanged
• No patient-identifiable document workflow was introduced or expanded
• Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy) — no Supabase/env/RPC/migration change
• Service-role keys and private document access remain server-only — unchanged
• Demo/synthetic content remains clearly separated from real clinical sources — unchanged
• Source metadata, review status, and outdated/unknown-source behavior remain conservative — no source-governance metadata was added to ranking/ordering; candidate scores untouched
• Deployment classification/TGA SaMD impact checked — this is a defensive prompt-hardening change; it strengthens (does not weaken) grounding/refusal behavior for clinical decision support

Notes

• Deferred follow-ups (tracked in threat-model §6): widening the neutralizer vocabulary (mitigation Bump gitleaks/gitleaks-action from 2 to 3 #8 — high false-positive risk on clinical prose, and the provenance boundary already covers the class); the seeded-chunk generation eval harness (Improve Clinical KB dashboard and RAG hardening #11) needed for answer-text-level INJ cases (INJ-1/2/5/9/10/11/14/15); per-chunk numeric scoping (Bump lucide-react from 1.16.0 to 1.17.0 #9); cross-source dose-disagreement flag (Improve Clinical KB design and RAG hardening #10); source-authority tier / prose-entailment / structural-role separation (Align project dependencies and cleanup stale branches #12–14).
• Observability gap surfaced by the measurement: production rag_retrieval_logs does not persist per-answer route/model/token usage, so answer cost is only measurable via eval:rag (a Phase-7 metric worth adding separately).

🤖 Generated with Claude Code

https://claude.ai/code/session_01VdSevAcjHqHt4ezZkWYbE7

Generated by Claude Code